Move files_external mount config to the database #20370

icewind1991 · 2015-11-06T14:33:05Z

TODO:

"global" availability
object storage config and other complex structures as storage config
migration
migration tests
"advanced" auth (External storage 'Login credentials' auth mechanism #18531)

cc @PVince81 @Xenopathic

DeepDiver1975 · 2015-11-06T14:49:22Z

apps/files_external/appinfo/database.xml

+		<name>*dbprefix*external_mounts</name>
+		<declaration>
+			<field>
+				<name>mount_id</name>


id is enough

I prefer to keep the column names a bit more descriptive to make the queries more readable

icewind1991 · 2015-11-23T14:55:40Z

Added migration, no tests for now, not sure how to do it bests.

@PVince81 @Xenopathic feel like reviewing 😄

icewind1991 · 2015-11-23T14:56:41Z

Currently migration leaves the original mount.json files in tact.

Once everything is properly tested I would like to rename them to something like mount.json.backup

I don't want to remove them since that would remove any chance of (manual) recovery in case of migration errors

MorrisJobke · 2015-11-25T12:35:06Z

@icewind1991 Update also the title and the labels if this is ready for review. Thanks :)

PVince81 · 2015-11-25T12:46:49Z

Once everything is properly tested I would like to rename them to something like mount.json.backup

@icewind1991 sounds good

icewind1991 · 2015-11-27T16:22:55Z

I think this is ready for review, there's still work to be done but it shouldn't cause any regressions and it's easier to work on the remaining stuff in smaller bits

PVince81 · 2015-12-01T15:04:18Z

apps/files_external/service/legacystoragesservice.php

+			$applicableGroups[] = $applicable;
+			$storageConfig->setApplicableGroups($applicableGroups);
+		}
+		return $storageConfig;


I assume this was simply moved from the existing code

PVince81 · 2015-12-01T15:11:08Z

The code looks good, missing a few PHPDocs for clarification.
I'm a bit surprised about the amount of code and gymnastics needed with the config, but this is understandable because storage configs have multiple dimensions.

👍

PVince81 · 2015-12-01T15:11:50Z

@icewind1991 please set to "To review" after you updated according to the comments.

icewind1991 · 2015-12-01T15:11:58Z

Added phpdocs

PVince81 · 2015-12-02T12:29:06Z

Needs a second reviewer @Xenopathic @DeepDiver1975

PVince81 · 2015-12-03T12:32:49Z

Files external test plan by @davitol owncloud/QA#39

Please note that for this PR only the storage configuration needs to be tested (with updates), no need to test all operations of the storages themselves. As long as the storage is mounted properly and upload works, then it's fine.

davitol · 2015-12-04T12:45:43Z

👍

PVince81 · 2015-12-04T12:46:49Z

Oh, there's an Oracle issue...
Let me quickly rebase this to refire it.

PVince81 · 2015-12-04T13:25:29Z

Oracle fail:

08:13:45 .............................................E..PHP Warning:  call_user_func() expects parameter 1 to be a valid callback, class '\OCA\Files_Sharing\Hooks' not found in /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php on line 104
08:22:57 PHP Stack trace:
08:22:57 PHP   1. {main}() /usr/local/bin/phpunit:0
08:22:57 PHP   2. PHPUnit_TextUI_Command::main() /usr/local/bin/phpunit:537
08:22:57 PHP   3. PHPUnit_TextUI_Command->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:105
08:22:57 PHP   4. PHPUnit_TextUI_TestRunner->doRun() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:153
08:22:57 PHP   5. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php:406
08:22:57 PHP   6. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:57 PHP   7. call_user_func:{phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727}() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727
08:22:57 PHP   8. OCA\Files_Sharing\Tests\TestCase::tearDownAfterClass() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727
08:22:57 PHP   9. OC_User::deleteUser() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/apps/files_sharing/tests/testcase.php:114
08:22:57 PHP  10. OC\User\User->delete() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/user.php:231
08:22:57 PHP  11. OC\Hooks\PublicEmitter->emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/user/user.php:195
08:22:57 PHP  12. OC\Hooks\BasicEmitter->emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/publicemitter.php:32
08:22:57 PHP  13. call_user_func_array:{/ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98
08:22:57 PHP  14. OC\Server->OC\{closure}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98
08:22:57 PHP  15. OC_Hook::emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/server.php:211
08:22:57 PHP  16. call_user_func:{/ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php:104}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php:104
08:22:57 PHP Warning:  call_user_func() expects parameter 1 to be a valid callback, class '\OCA\Files_Sharing\Hooks' not found in /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php on line 104
08:22:57 PHP Stack trace:
08:22:57 PHP   1. {main}() /usr/local/bin/phpunit:0
08:22:57 PHP   2. PHPUnit_TextUI_Command::main() /usr/local/bin/phpunit:537
08:22:57 PHP   3. PHPUnit_TextUI_Command->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:105
08:22:57 PHP   4. PHPUnit_TextUI_TestRunner->doRun() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:153
08:22:57 PHP   5. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php:406
08:22:57 PHP   6. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:57 PHP   7. call_user_func:{phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727}() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727
08:22:57 PHP   8. OCA\Files_Sharing\Tests\TestCase::tearDownAfterClass() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727
08:22:57 PHP   9. OC_User::deleteUser() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/apps/files_sharing/tests/testcase.php:115
08:22:57 PHP  10. OC\User\User->delete() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/user.php:231
08:22:57 PHP  11. OC\Hooks\PublicEmitter->emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/user/user.php:195
08:22:57 PHP  12. OC\Hooks\BasicEmitter->emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/publicemitter.php:32
08:22:57 PHP  13. call_user_func_array:{/ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98
08:22:57 PHP  14. OC\Server->OC\{closure}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98
08:22:57 PHP  15. OC_Hook::emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/server.php:211
08:22:57 PHP  16. call_user_func:{/ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php:104}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php:104
08:22:57 PHP Warning:  call_user_func() expects parameter 1 to be a valid callback, class '\OCA\Files_Sharing\Hooks' not found in /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php on line 104
08:22:57 PHP Stack trace:
08:22:57 PHP   1. {main}() /usr/local/bin/phpunit:0
08:22:57 PHP   2. PHPUnit_TextUI_Command::main() /usr/local/bin/phpunit:537
08:22:57 PHP   3. PHPUnit_TextUI_Command->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:105
08:22:57 PHP   4. PHPUnit_TextUI_TestRunner->doRun() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:153
08:22:57 PHP   5. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php:406
08:22:57 PHP   6. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:57 PHP   7. call_user_func:{phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727}() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727
08:22:57 PHP   8. OCA\Files_Sharing\Tests\TestCase::tearDownAfterClass() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:727
08:22:57 PHP   9. OC_User::deleteUser() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/apps/files_sharing/tests/testcase.php:116
08:22:57 PHP  10. OC\User\User->delete() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/user.php:231
08:22:57 PHP  11. OC\Hooks\PublicEmitter->emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/user/user.php:195
08:22:57 PHP  12. OC\Hooks\BasicEmitter->emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/publicemitter.php:32
08:22:57 PHP  13. call_user_func_array:{/ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98
08:22:57 PHP  14. OC\Server->OC\{closure}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hooks/emittertrait.php:98
08:22:57 PHP  15. OC_Hook::emit() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/server.php:211
08:22:57 PHP  16. call_user_func:{/ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php:104}() /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/lib/private/hook.php:104
08:22:57 PHP Fatal error:  Call to a member function getFileInfo() on null in /ssd/jenkins/workspace/core-ci-linux/database/oci/label/SLAVE/apps/files_sharing/tests/permissions.php on line 116
08:22:57 PHP Stack trace:
08:22:57 PHP   1. {main}() /usr/local/bin/phpunit:0
08:22:57 PHP   2. PHPUnit_TextUI_Command::main() /usr/local/bin/phpunit:537
08:22:57 PHP   3. PHPUnit_TextUI_Command->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:105
08:22:57 PHP   4. PHPUnit_TextUI_TestRunner->doRun() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:153
08:22:57 PHP   5. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php:406
08:22:57 PHP   6. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:57 PHP   7. PHPUnit_Framework_TestCase->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:57 PHP   8. PHPUnit_Framework_TestResult->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php:699
08:22:57 PHP   9. PHP_Invoker->invoke() phar:///usr/local/bin/phpunit/phpunit/Framework/TestResult.php:607
08:22:57 PHP  10. call_user_func_array:{phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93}() phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93
08:22:57 PHP  11. PHPUnit_Framework_TestCase->runBare() phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93
08:22:57 PHP  12. Test_Files_Sharing_Permissions->tearDown() phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php:774
08:22:57 PHP Warning:  Unknown: Cannot destroy the zip context in Unknown on line 0
08:22:57 PHP Stack trace:
08:22:57 PHP   1. {main}() /usr/local/bin/phpunit:0
08:22:57 PHP   2. PHPUnit_TextUI_Command::main() /usr/local/bin/phpunit:537
08:22:57 PHP   3. PHPUnit_TextUI_Command->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:105
08:22:57 PHP   4. PHPUnit_TextUI_TestRunner->doRun() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:153
08:22:57 PHP   5. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php:406
08:22:57 PHP   6. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:57 PHP   7. PHPUnit_Framework_TestCase->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:57 PHP   8. PHPUnit_Framework_TestResult->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php:699
08:22:57 PHP   9. PHP_Invoker->invoke() phar:///usr/local/bin/phpunit/phpunit/Framework/TestResult.php:607
08:22:57 PHP  10. call_user_func_array:{phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93}() phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93
08:22:57 PHP  11. PHPUnit_Framework_TestCase->runBare() phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93
08:22:57 PHP  12. Test_Files_Sharing_Permissions->tearDown() phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php:774
08:22:57 PHP Warning:  Unknown: Cannot destroy the zip context in Unknown on line 0
08:22:57 PHP Stack trace:
08:22:57 PHP   1. {main}() /usr/local/bin/phpunit:0
08:22:58 PHP   2. PHPUnit_TextUI_Command::main() /usr/local/bin/phpunit:537
08:22:58 PHP   3. PHPUnit_TextUI_Command->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:105
08:22:58 PHP   4. PHPUnit_TextUI_TestRunner->doRun() phar:///usr/local/bin/phpunit/phpunit/TextUI/Command.php:153
08:22:58 PHP   5. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/TextUI/TestRunner.php:406
08:22:58 PHP   6. PHPUnit_Framework_TestSuite->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:58 PHP   7. PHPUnit_Framework_TestCase->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestSuite.php:722
08:22:58 PHP   8. PHPUnit_Framework_TestResult->run() phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php:699
08:22:58 PHP   9. PHP_Invoker->invoke() phar:///usr/local/bin/phpunit/phpunit/Framework/TestResult.php:607
08:22:58 PHP  10. call_user_func_array:{phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93}() phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93
08:22:58 PHP  11. PHPUnit_Framework_TestCase->runBare() phar:///usr/local/bin/phpunit/php-invoker/Invoker.php:93
08:22:58 PHP  12. Test_Files_Sharing_Permissions->tearDown() phar:///usr/local/bin/phpunit/phpunit/Framework/TestCase.php:774

PVince81 · 2015-12-04T13:26:33Z

Doesn't look directly related, but also doesn't seem to fail on master.

icewind1991 · 2015-12-04T13:38:13Z

https://ci.owncloud.org/job/core-ci-linux/database=oci,label=SLAVE/13045/ says oci passed...

PVince81 · 2015-12-04T13:42:48Z

Okay, maybe it was the old one... I clicked on Details while it was still running.

Move files_external mount config to the database

LukasReschke · 2016-02-24T13:24:55Z

This probably broke ownCloud mounts: #22605

lock · 2019-08-07T13:01:52Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

icewind1991 added the 2 - Developing label Nov 6, 2015

icewind1991 added this to the 9.0-current milestone Nov 6, 2015

DeepDiver1975 added the in progress label Nov 6, 2015

icewind1991 mentioned this pull request Nov 6, 2015

Move mount.json to database configuration #20354

Closed

3 tasks

DeepDiver1975 reviewed Nov 6, 2015
View reviewed changes

icewind1991 force-pushed the external-db-config branch from cb18d1a to 8515031 Compare November 9, 2015 13:19

PVince81 added the app:files_external label Nov 16, 2015

icewind1991 force-pushed the external-db-config branch 2 times, most recently from bdcf061 to df53bc7 Compare November 23, 2015 14:06

icewind1991 mentioned this pull request Nov 24, 2015

Add command to get and set config for external mounts #20710

Merged

icewind1991 force-pushed the external-db-config branch from 5fe7207 to f302c79 Compare November 25, 2015 11:53

PVince81 mentioned this pull request Nov 26, 2015

Add table/value to map user id to storage id #15166

Closed

icewind1991 force-pushed the external-db-config branch from f302c79 to 47ad430 Compare November 27, 2015 16:17

icewind1991 changed the title ~~[WIP] Move files_external mount config to the database~~ Move files_external mount config to the database Nov 27, 2015

icewind1991 added 3 - To Review and removed 2 - Developing labels Nov 27, 2015

PVince81 reviewed Dec 1, 2015
View reviewed changes

icewind1991 added 3 - To Review and removed 2 - Developing labels Dec 1, 2015

PVince81 added 4 - To release and removed 3 - To Review labels Dec 4, 2015

PVince81 added 3 - To Review and removed 4 - To release labels Dec 4, 2015

db config backend for files_external

a1898dc

PVince81 force-pushed the external-db-config branch from 88852bb to a1898dc Compare December 4, 2015 12:48

PVince81 pushed a commit that referenced this pull request Dec 4, 2015

Merge pull request #20370 from owncloud/external-db-config

e91eba4

Move files_external mount config to the database

PVince81 merged commit e91eba4 into master Dec 4, 2015

DeepDiver1975 removed the in progress label Dec 4, 2015

PVince81 deleted the external-db-config branch December 4, 2015 13:42

PVince81 mentioned this pull request Dec 7, 2015

Storage migrator must not throw when no user exists / cron error #21010

Closed

lock bot locked as resolved and limited conversation to collaborators Aug 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move files_external mount config to the database #20370

Move files_external mount config to the database #20370

icewind1991 commented Nov 6, 2015

DeepDiver1975 Nov 6, 2015

icewind1991 Nov 25, 2015

icewind1991 commented Nov 23, 2015

icewind1991 commented Nov 23, 2015

MorrisJobke commented Nov 25, 2015

PVince81 commented Nov 25, 2015

icewind1991 commented Nov 27, 2015

PVince81 Dec 1, 2015

icewind1991 Dec 1, 2015

PVince81 commented Dec 1, 2015

PVince81 commented Dec 1, 2015

icewind1991 commented Dec 1, 2015

PVince81 commented Dec 2, 2015

PVince81 commented Dec 3, 2015

davitol commented Dec 4, 2015

PVince81 commented Dec 4, 2015

PVince81 commented Dec 4, 2015

PVince81 commented Dec 4, 2015

icewind1991 commented Dec 4, 2015

PVince81 commented Dec 4, 2015

LukasReschke commented Feb 24, 2016

lock bot commented Aug 7, 2019

Move files_external mount config to the database #20370

Move files_external mount config to the database #20370

Conversation

icewind1991 commented Nov 6, 2015

DeepDiver1975 Nov 6, 2015

Choose a reason for hiding this comment

icewind1991 Nov 25, 2015

Choose a reason for hiding this comment

icewind1991 commented Nov 23, 2015

icewind1991 commented Nov 23, 2015

MorrisJobke commented Nov 25, 2015

PVince81 commented Nov 25, 2015

icewind1991 commented Nov 27, 2015

PVince81 Dec 1, 2015

Choose a reason for hiding this comment

icewind1991 Dec 1, 2015

Choose a reason for hiding this comment

PVince81 commented Dec 1, 2015

PVince81 commented Dec 1, 2015

icewind1991 commented Dec 1, 2015

PVince81 commented Dec 2, 2015

PVince81 commented Dec 3, 2015

davitol commented Dec 4, 2015

PVince81 commented Dec 4, 2015

PVince81 commented Dec 4, 2015

PVince81 commented Dec 4, 2015

icewind1991 commented Dec 4, 2015

PVince81 commented Dec 4, 2015

LukasReschke commented Feb 24, 2016

lock bot commented Aug 7, 2019